Phrase Discovery for English and Cross-language Retrieval at TREC 6

Berkeley's experiments in TREC-6 center around phrase discovery in topics and documents. The technique of ranking bigram term pairs by their expected mutual information value was utilized for English phrase discovery as well as Chinese seg-mentation. This diierentiates our phrase-nding method from the mechanistic one of using all bigrams which appear at least 25 times in the collection. Phrase nd-ing presents an interesting interaction with stop words and stop word processing. English phrase discovery proved very important in a dictionary-based English to German cross language run. Our participation in the ltering track was marked with an interesting strictly Boolean retrieval as well as some experimentation with maximum utility thresholds on probabilistically ranked retrieval.